Fix bench-compare workflow for fork PRs by Copilot · Pull Request #182 · ember-tooling/ember-eslint-parser

Copilot · 2026-03-18T02:00:51Z

The bench-compare workflow fails on fork PRs because ref: ${{ github.head_ref }} resolves to a branch name that only exists on the fork remote, not on origin. Confirmed via PR #181 from NullVoxPopuli-ai-agent/ember-eslint-parser.

Use commit SHA instead of branch name for checkout — github.event.pull_request.head.sha resolves correctly for both same-repo and fork PRs
Guard post-bench steps on checkout success — Format PR comment, Write job summary, and Post PR comment previously ran with if: always(), causing secondary failures when checkout failed and no files existed

# Before
ref: ${{ github.head_ref }}

# After
ref: ${{ github.event.pull_request.head.sha }}

📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

github-actions · 2026-03-18T02:11:17Z

🏎️ Benchmark Comparison

	Benchmark	Control (avg)	Experiment (avg)	Δ
🟡	gts small	2.36 ms	2.28 ms	-3.2%
🔴	gts medium	8.13 ms	8.61 ms	+5.9%
🟡	gts large	20.17 ms	19.51 ms	-3.3%
🟡	gjs small	1.63 ms	1.68 ms	+3.0%
🟡	gjs medium	6.48 ms	6.32 ms	-2.5%
🟡	gjs large	15.62 ms	15.37 ms	-1.6%
🟡	hbs small	190.17 µs	186.43 µs	-2.0%
🟡	hbs medium	2.90 ms	2.86 ms	-1.2%
⚪	hbs large	14.77 ms	14.86 ms	+0.6%

🟢 faster · 🔴 slower · 🟡 within 5% · ⚪ within 1%

Full mitata output

clk: ~3.36 GHz
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
runtime: node 24.14.0 (x64-linux)

benchmark                   avg (min … max) p75 / p99    (min … top 1%)
------------------------------------------- -------------------------------
gts small (control)            2.36 ms/iter   2.61 ms ██▃                  
                       (1.15 ms … 13.52 ms)   7.09 ms ███▅▃                
                    ( 14.33 kb …   2.39 mb) 640.16 kb ██████▃▃▄▅▄▅▄▃▂▁▁▁▁▁▁

gts small (experiment)         2.28 ms/iter   2.75 ms ▅█                   
                       (1.16 ms … 12.16 ms)   7.03 ms ██▂                  
                    ( 76.34 kb …   1.96 mb) 641.29 kb █████▅▄▃▃▅▃▄▄▂▂▁▁▁▁▁▂

                             ┌                                            ┐
                             ╷ ┌──────┬─┐                                 ╷
         gts small (control) ├─┤      │ ├─────────────────────────────────┤
                             ╵ └──────┴─┘                                 ╵
                             ╷┌───────┬──┐                                ╷
      gts small (experiment) ├┤       │  ├────────────────────────────────┤
                             ╵└───────┴──┘                                ╵
                             └                                            ┘
                             1.15 ms            4.12 ms             7.09 ms

summary
  gts small (experiment)
   1.03x faster than gts small (control)

------------------------------------------- -------------------------------
gts medium (control)           8.13 ms/iter   9.02 ms  █                   
                       (4.09 ms … 31.66 ms)  28.14 ms  █▅▃                 
                    (685.59 kb …  10.93 mb)   2.72 mb ▇███▆▄▄▃▃▁▃▂▁▂▁▁▁▁▁▁▂

gts medium (experiment)        8.61 ms/iter  10.19 ms  █▆                  
                       (4.37 ms … 31.89 ms)  27.91 ms  ██▃                 
                    (155.54 kb …  10.30 mb)   2.68 mb ████▃▅▇▇▃▂▁▂▁▁▂▁▁▁▁▁▂

                             ┌                                            ┐
                             ╷ ┌─────┬┐                                   ╷
        gts medium (control) ├─┤     │├───────────────────────────────────┤
                             ╵ └─────┴┘                                   ╵
                              ╷ ┌────┬──┐                                 ╷
     gts medium (experiment)  ├─┤    │  ├─────────────────────────────────┤
                              ╵ └────┴──┘                                 ╵
                             └                                            ┘
                             4.09 ms           16.12 ms            28.14 ms

summary
  gts medium (control)
   1.06x faster than gts medium (experiment)

------------------------------------------- -------------------------------
gts large (control)           20.17 ms/iter  20.71 ms   █ ▄                
                      (12.00 ms … 46.35 ms)  44.49 ms  ██ █▅               
                    (  4.20 mb …   8.29 mb)   6.96 mb ██████▅█▁▁▁▁▅▁▁▅▁▁▁▁▅

gts large (experiment)        19.51 ms/iter  20.38 ms   ▃▃█                
                      (10.39 ms … 46.09 ms)  41.77 ms ▂▂███▂▇▂             
                    (  4.03 mb …   8.24 mb)   6.82 mb ████████▆▁▆▁▁▆▁▆▁▁▁▁▆

                             ┌                                            ┐
                               ╷   ┌──────┬┐                              ╷
         gts large (control)   ├───┤      │├──────────────────────────────┤
                               ╵   └──────┴┘                              ╵
                             ╷    ┌──────┬┐                           ╷
      gts large (experiment) ├────┤      │├───────────────────────────┤
                             ╵    └──────┴┘                           ╵
                             └                                            ┘
                             10.39 ms           27.44 ms           44.49 ms

summary
  gts large (experiment)
   1.03x faster than gts large (control)

------------------------------------------- -------------------------------
gjs small (control)            1.63 ms/iter   1.54 ms █                    
                     (880.84 µs … 18.39 ms)   6.69 ms ██                   
                    ( 58.03 kb …   6.27 mb) 567.29 kb ██▅▄▃▂▂▂▁▂▂▃▁▁▁▂▁▁▁▁▁

gjs small (experiment)         1.68 ms/iter   1.46 ms ▆█                   
                     (919.34 µs … 25.44 ms)   6.57 ms ██                   
                    ( 58.02 kb …   2.37 mb) 555.40 kb ███▃▃▃▂▂▂▂▃▃▂▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ╷┌────┬                                      ╷
         gjs small (control) ├┤    │──────────────────────────────────────┤
                             ╵└────┴                                      ╵
                             ╷┌────┬                                     ╷
      gjs small (experiment) ├┤    │─────────────────────────────────────┤
                             ╵└────┴                                     ╵
                             └                                            ┘
                             880.84 µs           3.78 ms            6.69 ms

summary
  gjs small (control)
   1.03x faster than gjs small (experiment)

------------------------------------------- -------------------------------
gjs medium (control)           6.48 ms/iter   6.96 ms █                    
                       (3.47 ms … 35.07 ms)  33.46 ms █▃▃                  
                    (559.48 kb …   3.70 mb)   2.26 mb ███▅▅▃▃▂▁▁▁▁▁▁▁▂▁▁▁▁▂

gjs medium (experiment)        6.32 ms/iter   7.07 ms ▄█                   
                       (3.21 ms … 33.23 ms)  25.59 ms ██ ▅                 
                    (702.93 kb …  11.76 mb)   2.35 mb ████▆▆▃▄▃▁▂▁▂▁▁▁▁▁▁▁▂

                             ┌                                            ┐
                             ╷┌───┬┐                                      ╷
        gjs medium (control) ├┤   │├──────────────────────────────────────┤
                             ╵└───┴┘                                      ╵
                             ╷┌───┬┐                          ╷
     gjs medium (experiment) ├┤   │├──────────────────────────┤
                             ╵└───┴┘                          ╵
                             └                                            ┘
                             3.21 ms           18.33 ms            33.46 ms

summary
  gjs medium (experiment)
   1.03x faster than gjs medium (control)

------------------------------------------- -------------------------------
gjs large (control)           15.62 ms/iter  17.04 ms   █ ▂                
                       (8.00 ms … 43.38 ms)  37.19 ms ▂▂█▇█  ▅             
                    (  3.72 mb …   7.31 mb)   6.21 mb █████▇▄█▄▄▁▁▁▁▁▁▄▄▁▁▄

gjs large (experiment)        15.37 ms/iter  16.51 ms  ▅ ▅█ ▅              
                       (8.10 ms … 45.68 ms)  36.65 ms ▃█ ██ █              
                    (  4.79 mb …  16.78 mb)   6.45 mb █████▄███▄▁▁▁▁▁▁▁▁▄▁▄

                             ┌                                            ┐
                             ╷   ┌───────┬─┐                              ╷
         gjs large (control) ├───┤       │ ├──────────────────────────────┤
                             ╵   └───────┴─┘                              ╵
                             ╷    ┌─────┬─┐                              ╷
      gjs large (experiment) ├────┤     │ ├──────────────────────────────┤
                             ╵    └─────┴─┘                              ╵
                             └                                            ┘
                             8.00 ms           22.59 ms            37.19 ms

summary
  gjs large (experiment)
   1.02x faster than gjs large (control)

------------------------------------------- -------------------------------
hbs small (control)          190.17 µs/iter 114.49 µs █                    
                      (81.18 µs … 25.56 ms)   2.57 ms █                    
                    (  6.94 kb …   1.94 mb) 139.00 kb █▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁

hbs small (experiment)       186.43 µs/iter 107.83 µs █                    
                      (77.96 µs … 23.29 ms)   2.62 ms █                    
                    ( 10.21 kb …   1.82 mb) 137.54 kb █▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ┌─┬                                         ╷
         hbs small (control) │ │─────────────────────────────────────────┤
                             └─┴                                         ╵
                             ┌─┬                                          ╷
      hbs small (experiment) │ │──────────────────────────────────────────┤
                             └─┴                                          ╵
                             └                                            ┘
                             77.96 µs            1.35 ms            2.62 ms

summary
  hbs small (experiment)
   1.02x faster than hbs small (control)

------------------------------------------- -------------------------------
hbs medium (control)           2.90 ms/iter   2.86 ms █                    
                       (1.34 ms … 24.66 ms)  21.39 ms █                    
                    (398.00 kb …   3.79 mb)   1.98 mb █▆▃▆▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁

hbs medium (experiment)        2.86 ms/iter   2.98 ms █                    
                       (1.37 ms … 21.12 ms)  20.90 ms █                    
                    (583.70 kb …   3.67 mb)   1.98 mb █▅▃▅▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ┌──┬                                         ╷
        hbs medium (control) │  │─────────────────────────────────────────┤
                             └──┴                                         ╵
                             ┌──┬┐                                       ╷
     hbs medium (experiment) │  │├───────────────────────────────────────┤
                             └──┴┘                                       ╵
                             └                                            ┘
                             1.34 ms           11.37 ms            21.39 ms

summary
  hbs medium (experiment)
   1.01x faster than hbs medium (control)

------------------------------------------- -------------------------------
hbs large (control)           14.77 ms/iter  17.20 ms █                    
                       (8.36 ms … 35.44 ms)  33.52 ms █▇▅  ▂               
                    (  7.46 mb …  10.55 mb)   9.10 mb ████▆█▁▆▃▁▁▁▁▆▁▆▃▁▁▁▆

hbs large (experiment)        14.86 ms/iter  15.41 ms █▆                   
                       (8.30 ms … 36.21 ms)  33.88 ms ██                   
                    (  7.10 mb …  10.84 mb)   9.07 mb ██▅▅▅▇▃▁▃▁▃▃▁▃▁▇▁▃▃▁▃

                             ┌                                            ┐
                             ╷ ┌────────┬────┐                           ╷
         hbs large (control) ├─┤        │    ├───────────────────────────┤
                             ╵ └────────┴────┘                           ╵
                             ╷┌──────────┬┐                               ╷
      hbs large (experiment) ├┤          │├───────────────────────────────┤
                             ╵└──────────┴┘                               ╵
                             └                                            ┘
                             8.30 ms           21.09 ms            33.88 ms

summary
  hbs large (control)
   1.01x faster than hbs large (experiment)

Use github.event.pull_request.head.sha instead of github.head_ref for the checkout ref, since head_ref is a branch name that only exists on the fork remote and causes checkout to fail for fork PRs. Also guard post-bench steps with steps.checkout.outcome == 'success' so they don't fail when checkout itself fails. Co-authored-by: NullVoxPopuli <[email protected]>

Initial plan

97d3ba6

Copilot AI assigned Copilot and NullVoxPopuli Mar 18, 2026

Copilot started work on behalf of NullVoxPopuli March 18, 2026 02:00 View session

Copilot AI changed the title ~~[WIP] Fix the bench script for GH~~ Fix bench-compare workflow for fork PRs Mar 18, 2026

Copilot AI requested a review from NullVoxPopuli March 18, 2026 02:13

Copilot finished work on behalf of NullVoxPopuli March 18, 2026 02:13

NullVoxPopuli marked this pull request as ready for review March 18, 2026 02:15

NullVoxPopuli merged commit 65c81c0 into main Mar 18, 2026
34 checks passed

NullVoxPopuli added the internal label Mar 18, 2026

github-actions Bot mentioned this pull request Mar 19, 2026

Prepare Release v0.9.0 #175

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bench-compare workflow for fork PRs#182

Fix bench-compare workflow for fork PRs#182
NullVoxPopuli merged 2 commits intomainfrom
copilot/fix-bench-script-for-gh

Copilot AI commented Mar 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🏎️ Benchmark Comparison

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Mar 18, 2026 •

edited

Loading

github-actions Bot commented Mar 18, 2026 •

edited

Loading